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1.  Statement  of  the  Problem  Studied 

This  research  supported  by  this  grant  involves  development  and  study  of  algorithms  for  the  numeri¬ 
cal  solution  of  large-scale  nonlinear  optimization  problems.  Optimization  problems  constitute  an  impor¬ 
tant  class  of  mathematical  problems  whose  computer  solution  is  desired  by  scientists  and  engineers.  The 
need  to  solve  such  problems  arises  in  a  huge  variety  of  computer  simulations  and  data  analyses,  with 
examples  including  optimal  design  of  aircraft  and  spacecraft  for  low  cost  or  high  efficiency,  optimal  con¬ 
trol  of  chemical  processes,  and  calculation  of  the  native  states  of  biological  molecules.  In  recent  years,  as 
computer  power  has  increased,  the  feasibility  of  investigating  optimal  designs  has  increased,  and  along 
with  this  the  need  for  improved  optimization  methods  has  grown.  This  requires  fundamental  research  in 
optimization  algorithms,  especially  for  classes  of  large-scale  problems. 

In  general,  research  in  optimization  algorithms  has  two  strategic  branches.  One  type  of  research  focusses 
on  broad  purpose  algorithms  that  can  be  ultimately  incorporated  into  general  purpose  algorithms  that  can 
solve  most  problems  in  some  class.  Some  applications,  however,  generate  problems  that  require  special 
purpose  algorithms  that  are  too  difficult  for  such  general  methods  and  require  development  of  algorithms 
that  are  tailored  to  that  application.  The  research  funded  by  the  grant  was  in  areas  of  both  types.  We 
worked  on  successive  linear-quadratic  programming  algorithms  for  large-scale  constrained  optimization, 
and  on  tensor  methods  for  fast  solution  of  singular  problems.  Much  of  our  effort  was  on  the  application  of 
optimization  to  problems  in  molecular  chemistry  including  prediction  of  protein  structure  by  potential 
energy  minimization. 

One  of  the  main  emphases  of  this  research  program  in  recent  years  has  been  the  development  of 
global  optimization  methods  for  the  solution  of  large  molecular  configuration  problems.  The  objective  of 
this  research  is  to  develop  methods  that  are  reliably  capable  of  solving  difficult,  large  nonlinear  global 
optimization  problems.  Our  current  work  is  investigating  methods  to  find  the  native  configurations  of 
proteins  and  polymers.  This  is  a  problem  of  great  importance  in  science;  it  includes  the  well-known  pro¬ 
tein  folding  problem  as  well  as  the  investigation  of  polymers  that  are  used  to  make  new  materials.  The 
resultant  optimization  problems  are  very  difficult  because  they  have  extremely  large  numbers  of  local 
minimizers  that  have  similar  function  values.  In  addition,  some  of  the  techniques  developed  in  this  effort 
have  turned  out  to  be  of  more  general  applicability. 

Another  main  focus  of  our  research  is  the  development  of  new  algorithms  for  large-scale  uncon¬ 
strained  and  constrained  optimization  problems,  including  limited-memory  methods  for  problems  with 
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many  thousands  of  variables,  and  interior  point  methods  for  nonlinearly  constrained  problems.  We  were 
particularly  interested  in  methods  such  as  sucessive  linear-quadratic  programming  that  are  capable  of 
making  good  use  of  a  woarm  start.  We  are  also  investigating  theoretical  convergence  issues  for  optimiza¬ 
tion  algorithms  that  have  practical  consequences  in  how  problems  are  formulated,  and  new  approaches  to 
one  of  the  key  numerical  linear  algebra  subproblems  that  is  particular  to  optimization  algorithms. 


2.  Summary  of  the  Most  Important  Results 

A  major  focus  in  this  grant  has  been  the  development  and  testing  of  techniques  for  solving  global 
optimization  problems  for  determining  the  structure  of  proteins  and  polymers.  The  problem  is  to  find  the 
lowest  energy  configuration  of  a  protein  or  other  polymer.  This  problem  is  a  global  optimization  problem 
because  it  has  a  huge  number  of  local  minimizers.  In  addition,  locating  the  lowest  (global)  minimizer  is 
very  difficult.  For  proteins,  the  solution  of  this  problem  would  represent  a  solution  to  the  well-known 
protein-folding  problem. 

In  previous  years,  we  have  developed  a  stochastic/perturbation  approach  for  solving  global  opti¬ 
mization  problems  from  molecular  chemistry.  There  are  four  keys  to  this  approach.  The  first  is  a  large 
scale  global  optimization  methodology  that  performs  small-scale  global  optimizations  with  only  a  small 
number  of  parameters  variable  and  the  remaining  parameters  temporarily  fixed,  followed  by  local  mini¬ 
mizations  with  all  parameters  varying,  at  each  stage  of  the  global  optimization  procedure.  The  second  is 
the  incorporation  of  a  new,  efficient  approach  towards  smoothing  the  objective  function  in  the  global  opti¬ 
mization  framework.  Initially  these  have  been  the  backbone  of  the  approach.  More  recently  two  other 
aspects  have  become  key  to  doing  work  on  realistic  protein  targets.  One  of  these  is  the  incorporation  of 
predictions  from  secondary  structure  prediction  methods  in  the  initial  phase  of  our  algorithm,  to  produce 
starting  configurations  with  reasonably  good  secondary  structure.  The  final  one  is  work  with  our  chem¬ 
istry  partners  to  use  the  mismatch  between  simulation  and  experimental  evidence  to  continue  to  refine  the 
mathematical  energy  model  upon  which  our  global  optimization  approach  relies. 

By  now,  we  have  evolved  our  approach  to  be  able  to  handle  proteins  with  arbitrary  structure  and 
size.  The  crux  of  this  issue,  for  us  and  other  groups  doing  related  research,  is  the  ability  to  handle  beta- 
sheets.  Beta-sheets  are  the  other  main  type  of  secondary  structure  in  proteins.  However  they  are  far  less 
local  than  alpha-helices.  While  alpha-helices  are  continuous,  beta-sheets  are  formed  by  contiguous 
strands  that  can  be  arbitrarily  far  apart.  Secondary  structure  prediction  programs  can  predict  the  strands 
with  good  accuracy,  but  they  do  not  predict  which  strands  are  bonded  together,  nor  the  parallel  or  anti-par¬ 
allel  orientation  of  those  bonds.  Our  progress  in  this  area  was  tested  in  the  fifth  Critical  Assessment  of 
Techniques  for  Protein  Structure  Prediction  (CASP5)  competition,  which  started  in  summer  2002  with 
results  presented  in  December  2002.  Our  performance  in  the  "new  fold"  category,  for  which  our  method  is 
appropriate  ranked  approximately  18th  among  the  190  participating  groups. 

We  have  also  made  significant  progress  in  improving  the  local  minimization  phase  of  our  global  mini¬ 
mization  algorithm.  In  a  ran  of  this  method,  many  unconstrained  local  optimization  problems  must  be 
solved,  for  which  we  have  mainly  been  using  the  limited  memory  quasi-Newton  method  L-BFGS.  In  this 
period  we  have  focussed  on  improving  the  efficiency  of  this  local  optimization  phase  by  developing  a 
local  method  that  applies  a  preconditioner  to  the  the  limited  memory  BFGS  method.  The  preconditioner 
consists  of  an  approximation  to  part  of  Hessian  matrix  of  the  objective  function.  We  have  found  good 
ways  of  modifying  the  preconditioner  to  make  it  positive  definite.  The  effect  of  this  preconditioning  is  to 
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reduce  the  number  of  iterations  sufficiently  to  more  than  compensate  for  the  cost  of  computing  the  pre¬ 
conditioner.  Thus  we  have  been  able  to  reduce  the  total  time  of  a  local  minimization  by  a  factor  of  5  to  6. 
We  expect  this  improvement  to  have  a  significant  effect  on  the  global  optimization  procedure.  This  work 
has  been  described  in  a  paper  submitted  to  SIAM  Journal  on  Optimization. 

We  have  continued  our  work  on  methods  for  large-scale  nonlinearly  constrained  optimization  with  Jorge 
Nocedal  at  Northwestern  University.  Our  efforts  to  develop  a  method  based  on  using  successive  solution 
of  linear  programs  as  a  predictor  of  active  sets  has  been  fairly  successful  and  a  paper  describing  it  has 
been  published  by  Mathematical  Programming.  Although  this  method  is  not  necessarily  more  powerful 
than  primal-dual  interior  point  methods,  it  should  be  much  more  capable  of  making  good  use  of  a  warm 
starting  point.  This  is  important  in  problems  with  variable  sized  discretizations  and  in  branch  and  bound 
approaches.  We  have  been  able  to  establish  some  new  global  convergence  results  for  this  method  which 
have  been  described  in  a  paper  published  in  SIAM  Journal  on  Optimization.  We  have  also  developed  a 
software  implementation  of  this  method,  which  has  been  distributed  as  part  of  the  KNITRO  package. 

We  also  continued  work  on  tensor  methods  for  solving  large-scale  systems  of  nonlinear  equations. 
We  have  developed  a  "tensor-Krylov"  method  that  solves  a  local  tensor  model  to  within  a  specified  rela¬ 
tive  tolerance  using  the  techniques  associated  with  linear  Krylov  subspace  methods  and,  in  particular,  the 
Generalized  Minimum  Residual  method  (GMRES).  Our  tensor-Krylov  method  works  better  than  New- 
ton-GMRES  on  problems  that  have  an  ill-conditioned  or  singular  Jacobian  at  the  problem  root.  These 
types  of  problems  are  encountered  in  scientific  and  engineering  applications  that,  for  example,  exhibit 
shocks.  This  work  is  main  part  of  the  doctoral  dissertation  of  Brett  Bader  who  was  supported  on  this 
grant  and  is  now  at  Sandia  National  Laboratories.  Papers  on  this  work  have  been  published  in  SIAM 
Journal  on  Scientific  Computing  and  SIAM  Journal  on  Numerical  Analysis,  and  one  more  has  been  sub¬ 
mitted. 
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